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A project in Thailand to develop domain-referenced 
English language tests for use with Thai secondary and college 
students is described. The first phase consisted of developing 
standardized tests for listening, speaking, reading, and writing 
based on existing syllabuses, determining the common levels of 
English proficiency in those domains, and analyzing the communi -iv? 
competence factors in English use of the Thai students. Subjects were 
697 lower secondary, 525 upper secondary, and 493 university 
students. The second phase involved examining the consistency of 
domain specifications and setting domain-content standard scores for 
the tests. The process of test construction and validation are 
outlined, and results for each phase are reported. Recommendations 
for further research in this area include study of the contribution 
of individual language skills to the development of overall language 
proficiency, with the objective of finding effective remedial 
methods, and similar test and standard development for all levels of 
Thai education. Recommendations for practical application in English 
teaching and learning include use of a communicative context for 
language instruction and early emphasis on oral/aural skills 
gradually leading to later emphasis on reading and writing. A 55-item 
bibliography and test objectives, specifications, and analyses are 
appended. (KSE) 
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/ t TRODU CT I OM TO DOM A / 9-RBFEREMCED TEST IMG 

Doma i n- referenced testing dates back to 1966 when 
Wells Hively of the University of Minnesota started the 
M I NNEMAST Project a iming at the development of do^a in- 
referenced tests for science and mathematics. Publications on 
this subject coming* out in 1974 and 1975 were influential 
especially ir Minnesota to the widespread use of this new trend 
in testing. Domain-referenced testing expanded on the concept 
of criterion-referenced testing which measures students' 
mastery of the set objectives fay building items on the 
incorporated domain description* content limits, criterion of 
d i st ractor description, criterion of distractor domains, format 
and direction, and specimen items* Sen r ion and Rabehl (1974) 
delineated the steps of domain-referenced test construction as 
consisting of: 

1. developing the domain on the results of the 
analyses of content s/ sk ills, teaching and learning procedures, 
objectives and 1 earner- cent e red goals which are clear and 
easi ly understood, and 

2. developing domain-referenced test i tarns by 
sampling of t est ees and content domains, and setting suitable 
test techniques relevant to the teaching- 1 earning situations. 

The results of the tests are to be analyzed and 
reported to all parties concerned, i.e., teachers, students, 
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parents • etc. The report of the student's profile wi 1 1 f in turn* 
serve the purposes of remedial teaching and both short-term and 
long-term learning improvement. Information yielded by domain- 
referenced tests is greater than that yielded by criterion 

2 

referenced tests because it is directly linked to the domains 
of teaching and learning, thus, readily lending itself to 
remedial teaching and learning improvement. In other words, 
domain-referenced testing is a system developed to measure 
learning mastery which goes beyond criterion-referenced testing 
by not merely focusing on the criteria of achievement set by 
the behavioral objectives, but pinpointing also on the content 
/skill and the context of evaluation and the environment of 
learning and acquisition as a domain incorporates the areas 
covered by the objectives, the content of learning, the skills 
or sub-skill levels, for instance speaking skill at the 
criticism level, reading skill at the comprehension and 
interpretative levels. Student learning/testing activities, 
such as taking notes, listening to lectures, etc., and the 
learning/testing media, such as video--audio tapes, newspapers, 

OHP,etc. are included in the domain-description. A domain 
must have a definite area with a clear border 1 ine separating it 
from other domains. 

The amplified objectives on which a domain-referenced 
test specification is based are expanded from behavioral 
objectives which have been criticized as nnt being objective 
nor complex enough for measuring human learning. Specificity 
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is one of the necessary dimensions of amplified objectives* 
Roid and Haladyna ( 1982? 297) mentioned the effort to improve 
behavioral objectives by Popham of the University of California 
I OX ( I tern- Object ive Exchange) • Popham ' s smpl i f ied objectives 
cons i st s of 5 parts as f o 1 lows * 

1. Description of the traits to be measured or the 

3 

main objectives of testing, 

2# Specific objectives which have been transformed 
from the main objectives, 

3. Descriptions of item forms of both the stimuli 
and the responses; in other words, behaviors/performance to be 
tested, types and techniques of testing, the test items, format 
and techniques of testees's responses are specified, 

4. Cond i t ion and cr iter is of correct responses, 

5. Spec i men test i terns » 

Thorndike ( 1982: 2) described the process of 
deve loping domain-referenced tc st items as cons i st ing of 5 
steps as follows: 

1. Specify and limit the domain, 

2. Spec i f y the test techni ques or the format of the 

st imu 1 i , 

3. Specify the technique or mode of responses, 

4» Specify the amount of items unbiasedly sampled 
from the uni verse of content , 

5t Specify the criteria of mastery of the domain* 

The de s cr i pt i ons of doma in-referenced test ing and 
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amplified objectives above confirm their behavioral roots. 

* 

With the advent of individualized! learner-centered approach of 
the early 1970 f s, the domain-referenced item-form shall 
developed by Hively, et. al,(Alkin, 1973$ 25) incorporated the 
trans format 1 on ru 1 e by whi ch items can be subst i tut ad by 
paralleled ones* and the instructions to be given to individual 
students whose behaviors are to be observed and recorded* 
Lists of stimuli — verba 1 f pictorial f rea 1 ia , etc. --to be 

4 

presented to the testees form a component of an item shell. 

Purves , et . a 1 . ( 1984) spec i f i ed the system of gi ving 
instructions for ?ssay writing in their international study on 
domain-referenced essay writing to include 15 dimensions, i.e., 
instructions, stimuli , cognitive demand f tfoa 1 9 role, aud ience , 
consent, genre, style, advanced preparation of information, 
length, writing pattern, time, drafting and criterion. Purves * 
domain specification has proven useful for essay writing as it 
reminds the students of the detai 1 s and comp lex it i es of the 
writing tasks as well as serving as a tool for evaluating an essay* 
Communicative Approach to Dowain-rafmirencad Tasting 
Test ing commun i cat i ve 1 anguage competence is based on 
socio-cultural and pragmatic theories of language learning. 
Language contents, encoding skills and subs k ills — unitary or 
i nt egrat i ve — are meshed together in the dynamics of 
communication with the following communicative parameters* 
function, modality, channel of communication, status, role, 
style, topic and communicative situations. An item shell of a 
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communicative domain-referenced test may include discrete-point itejrs 
measuring linguistic competence^ macro/global items measuring 
soc i o 1 ingui st i c competence, s ingl e/uni t>ary and integrated- sk i 1 1 
items measuring strategic, pragmatic and discourse competence* 
Testing methods may be either or both selection (multiple 
choice, true/false, matching, sequencing, etc.) and supply 
(cloze, completion, short-answer, letter-writing, essay, etc*)* 

Using B • J . Carrol 1 1 s ( 1980) parameters for 
communicative test construction in developing a domain- 
referenced test the following parameters are included in the 

5 

test blueprint: 

1 . Context i time and locat ion of language 
interaction, e.g., in the classroom, at the train-station, in 
the restaurant t etc. 

2. Content Key: main idea or topic of language 
interaction, e.g., language in bargaining, in business 
transaction, in signing a treaty, in lecturing about danger of 
neuclear wars, in advertising, etc. 

3. Function and Discourse: language for work or 
personal social transaction^ elaborated/restricted code, 
referent i a 1 /d i rect i ve/poet i c/phat i c f unct i on or met a 1 anguage, 
descript ive/persuasive/expl anat or y d i scour se . 

4. Modality! language interactions via the oral mode 
(listening, speaking? or the visual mode (reading, writing). 

5. Genre : methods of employing language skills, e.g., 
writing a personal letter, writing an essay, taking notes, 
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writing a telegram, talking on the telephone, reading a 
newspaper, etc* 

6« Roles interaction between language transmitters 
and rece i vers , e.g., teacher- student , father - so?*, f r i end-f r i end , 
younger brother-elder sister, etc. 

7. Status: socio-economic statuses or professional 
areas governing the style and register of language use, e.g., 
ecclesiastical language, court language, vulgar language, 
language for specific purposes, such as English for airline 
hostesses and stewards and stewardesses, etc. 

8. Pre- suppos i t ions anticipation of language 
transmissions, e.g., expectancy of the incoming message through 

6 

some media or c face-to-face interaction! familiarity with the 
language user or transmitter will facilitate communication, for 
example reading a friend's smeared letter with some words or 
phrases missing, or listening to one's mother in a noisy market 
place* Here communication is not hampered on account of the 
receivers correct anticipation of the incoming message. 

9. Mood or Attitudes! the mood and attitudes of 
language users can determine the content and style of language 
use, e.g., humorous, rude, sincere, sarcastic, negative, 
pos it i ve, etc . 

10. Forma 1 i ty * the level of language use, e. g> , 
formal, colloquial, slang, academic, standard, sub-standard, etc. 

The af orement i oned framework for doma in-referenced 
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testing within the communicative language teaching/1 earning 
approach is congruent with the English language teaching 
/learning practice currently endorsed in various sectors of the 
Thai educational systems where English is viewed as the 
1 anguage of ? nternat i ona 1 communi cat i on and where meanings of 
the message are the heart of the matter* 

TBS RESEARCH 

The research on domain-referenced tests of English 
use was conducted on a grant given by the Thai government and 
was administratively supported by the Chu la 1 ongkorn University 
Language Institute and the Testing Bureau of the Ministry of 
Education. The research has two phases. Phase I 
Project was entitled Reearch for the Deve lopment of 

7 

Engl i sh Doroa in- Referenced Tests for Use with Tha i 
Students i n Var i ous Leve 1 s . The object ives of Phase I 
Project were to? 

! • Develop standardized domain-referenced test- 
items from the determined domains of English use via the sound 
modality, i.e., listening and speaking and the graphic modality, 
i.e., reading and writing, as specified in the English 
syllabuses for the lower secondary, upper secondary and 
tertiary levels of education, 

2. Determine the common levels of English 

proficiency in the specified aoma ins of students in the three 
levels of education as stated in objective # 1, and to 
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3 . Study , ana lyze and determine the factors of 
communi cat i ve competence in using English of Thai students In 
the lower secondary and tertiary levels of education. 

The mul t i -stage strst i f i ed random samp 1 in? method was 
einp 1 oyed to obta i n the samp 1 e groups of 697 1 ower secondary 
students , 525 upper secondary , and 493 Chul a longkorn 
Uni vers i ty students . 

The Rasch Model and the sensitivity and facility 
indices were used for test item analyses and revision. Domain- 
referenced methods of test analyses were used to establish the 
tests 1 coefficients of reliability and validity, F-t est s and t~ 
tests were used to establish test cr i ter i cn~re lated validity. 

Parametric statistics used to discover students 1 
levels of English proficiency in the specified domains of 
English use were arithmetic meant standard deviation 
coefficient of var i at i on( CV ) ♦ kurtosis and skewness. Students 1 

6 

proficiency levels were rated on a 5-band scales from very weak 
requiring extensive remedial work« to very good not 
requiring any remedial work • 

To study the factors of communi cat i ve competence 9 
Exploratory Principal Component Factor Analysis with Varimax 
Rotation and Pearson Product Moment Correlations were employed. 

The research project in Phase I! entitled 
Exam i nat i on of the Cons i stency of Doma i n Spec i f i cat i ons for Engl i sh 
Language Test i n the Lower and Upper Secondary Leve 1 s of Educat i on 
aimed at: 
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1* Examining the consistency .>f domain 

specifications of the domain-referenced tests of English use, and 

2* Setting the domain-content standard scores for 
the domain-referenced tests of English use constructed for the 
research* (Details in Appendix IV) 

The sample groups consisted of 563 lower secondary 
and 469 upper secondary students selected by the stratified 
random sampling method* 

Test reliability indices were established using the 
Cronbach Alpha Mehod* The Rasch Model was employed to examine 
the test items 9 difficulty indices and their congruence with 
the ICC or the item characteristic curve* To prove the 
consistency of domain specifications used to develop three 
forms of dome in-ref enced tests for each level totalling nine 
test s i the F-teet method was used on the arithmetic means of 
item difficulty of each form* With the F ratio of 0.00 and the 
probability of 0.99. the consistency of the domain 
specifications for the three parallel forms — Form A. Form B 9 

9 

and Form C was confirmed. 

The percentile method was used to set the test domain- 
content standard scores* (See Appendix IV) 

The research in both of its phases is a language 
testing research in its full sense by being* 

1* a research to develop three sets of domain- 
referenced tests of English use, three f^rms for each set • from 
the English curricula of the lower and upper levels of 
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secondary of education and from the English syllabuses of the 
Chulalongkorn University Language Institute for students in 
first, second and third years fct Chula longkorn University! the 
test items were processed through the various steps of test 
analyses for the purpose of test standardization and validation, 

2. a research to find out about th . nature and 
specific characteristics of communicative proficiency in the 
English language by the use of the constructed and standardized 
domain-referenced tests of English use, 

3. a research using the domain-referenced tests of 
English use as a tool for studying the levels of English 
ability of Thai students in three levels of education. 

LAMBUAGK TEST/MO RESEARCH IE TEE FIRST SEESE 

To develop, standardize, and validate the domain- 
referenced tfcsts of English use for the three levels of 
education — lower secondary, upper secondary, and tertiary — the 
following steps were taken* 

Step One 

From the learning objectives specified in the English 

10 

syllabuses amplified objectives were developed. 

(See Appendix I for a specimen set of amplified objectives) 
Step Two 

From the amplified objectives domain-referenced test 
tables of specification were developed to sample the English 
learning domains set in the syllabuses and encapsulated in the 
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ampl ! f led object i ves. 

( See Append ix I 1 for a specimen doma in- referenced 
test table of specification) 

St&p rAree 

The mult i trait-mult i met hod assessment scheme was used 
in test development • From the domain-referenced test tables of 
speci f icat ion the mult i method approach to test construct i on 
consisting of both selection-type items, ie. f multiple choice, 
matching, and item sequencing, and supply-type items, ie., 
completion of statements, diagrams and charts » short-answers, 
partial dictation, full dictation, summarization, paragraph 
writing and essays. The multitrait assessment consisted of 
measuring both language componants, ie., grammar , vocabulary, 
and phonology which are discrete-point items, and language skills! 
unitary skills of listening, speaking, reading and writing and 
integrative skills of listening-writing, 1 i etening-read ing- 
writing, and reading-writing. All the items were maximal 
performance tests except for the items measuring student 
speaking skill which employed a self-assessment rating scheme. 

For the lowt secondary level 225 items, 75 items per 
form, were constructed . Tor the upper secondary 1 eve 1 also 
225 items, 75 items per form, were developed. Fo*» the tertiary 

1 i 

level within the context of Chula longkorn University, 210 items , 
70 i terns per form, were constructed . The tests , therefore f 
cons isted of 3 parallel forms for each 1 eve 1 of educat i on . 
Each form was based on the domain-referenced test table of 
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specification developed from the amplified objectives for each 
1 eve 1 . 

Step Four 

For test standard i zat ion the f ol lowing' methods of 
test ana lys \ s were performed on the students 1 obta ined raw scores ft 

1 . The classical model of item ana lys is, ie. f Chung 
Teh Fan ( 1952) 27% upper-lower method was used to find out item 
difficulty and discrimination indices which were used as a 
basis for item revision. The Whitney & Sabers ( 1970* 1971) 
method was used on student scores from the mult iple-point items, 
ie*, essays , short -answers , partial dictation, full dictation, 
and summarization for the same purpose* 

2* The second pre~test s were conducted using the 
revised test forms on the sampled subjects of the first pre- 
tests consisting of 42 lower-secondary, 41 upper-secondary, and 
47 Chula longkorn University students. 

3 • The doma i n~ referenced mode 1 of i tern ana 1 ys i s was 
used to establish test-item sensitivity indices and facility 
va lues us ing the pre- post test data * Swam ina than, Hamb let on 
and Al gi na ' s (!n Piti yanuwat , 1986) methods of test ana 1 ys i s 
were used to find the reliability and validity indices of the 
tests* Item-objective congruence was also carried out to 
established the test content validity. 

12 

(See Appendix III for the formulae of test analyses) 

4. Test administration of the main sample groups for 
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the research consisting of 1,260 lower secondary, 994 upper 
secondary and 493 tertiary < Chuia longkorn University) students 
using the nine standardized and validated domain-referenced 
tests of English use was carried out. The data were analyzed 
using the Rasch Model to find item-fit statistics which 
indicated that over 95% of the items fit the ICC. 

Step Five 

To validate the tests the procrdures employed 

consisted of the following! 

1. As stated in tt 3 of Step Four the domain-referenced 
methods of test analysis were used to establish the tests' 
reliability and validity indices. 

2. The validity established through the item- 
objective congruence method was content validity. 

3. The tests' criterion-related validty was 
established through the pre-post test method mentioned in #3 
of Step Four. Besides the pre-post test method of test 
analysis, the tests ' cr i ter i on-re 1 ated validity was also 
established using t-tests for secondary students divided into 

2 groups on the basis of school type — small-medium sized, and 
large-very large sized, and F-tests for university students 
divided into 3 groups on the basis of their year of study-- 
first-year, second-yerr, and third-year. The t-values and 
the F-ratios which were statistically significant at .05 and .01 
levels confirmed the tests' criterion-related validity. 

13 
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4, The tests* construct validity was established 
using Pearson Product Moment Correlation method. Correlational 
matrices among the various parts of the tests using selection 
and supply item types and measuring different traits via 
unitary skill mode and integrative skill mode were analyzed. 
The correlations were labelled as follows; 

C » Monotra i t-Heteroinethod 

H = Heterotra it-Heteromet hod 

MM = Monotra it -Monomet hod 

V « Total score with part score 

M - Heterotra i t-Monomethod 
Correlations labelled C should be greater than those labelled M 
and MM which* in turn* should b& greater than correlations 
labelled MM if the tests are convergent ly va 1 i d . Besides* 
C and MM correlations should be greater than H if the tests are 
discriminant ly val id (Campbel 1 anu Fiske, 1967) . The V 
correlations should also be greater than the MM to prove the 
tests' discriminant validity. 

The correlational matrices for the three sets of 
domain-referenced tests exhibited these patterns of 
correlations indicating the tests' construct validity of the 
convergent and d i scr i m i nant types . 

Sfcep Six 

Doma i n~ content standard scores of the three tests were 
computed using the percentile methoa. Groups of items were 
identified from 5 to 95 percentiles. The grouping of the 
scores from each test exhibited similar patterns indicating the 
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consistency of item difficulty and student ability. 

14 

(See Appendix IV for Domain-Content Standard Scores) 
Step Seven 

The general i zabi 1 ity of the domain-referenced tables 
of specification was proven by F-tests of the item difficulty 
values. The F-ratio of 0.00 and the probability of 0.99 on all 
the tests proved that the ICC curves of the three parallel 
tests for each level did not statistically significantly 
deviate from the standard ICC. 

(See Appendix V for the F-tests of the domain- 
referenced tests' ICC *ith the standard ICC) 
LANGUAGE TEST I MO RESEARCH It TEE SECOMD SEMSE 

The research intended to find out about the nature 
and specific characteristics of communicative proficiency in 
English by using the constructed, standardized and validated 
domain-referenced tests of English use. The research 
procedures consisted oft 

Step One 

The analyses using Pearson Product Moment Correlation 
method revealed that language components, is. , phonology, 
vocabulary andgrammar, weremoderat e 1 y and significantly 
correlated with language modalities, ie., oral/aural and 
graphic/visual, and that the shared variances with the oral/aural o 
or sound modality were 19%, 35% and 24% for the lower secondary, 
upper secondary, and tertiary levels respectively. The variances 
shared with the visual modality were 22%, 24%, and 15% respectively. 
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Step Two 

Exploratory principal component factor analyses using 
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varimax rotation were performed on the data obtained through 
the mult itra it-mult imethod assessment scheme. For the lower 
secondary (students aged 14-15) four points emerged* 

1. At this level the sound modality was finely 
differentiated into 3 factors* Factor One --listening, word- 
stress, and speaking (SSQ 1.458), Factor Two— i ntegrat i ve 
skills of listening-reading-writing and sound discrimination (SSQ 
1.193) and Factor Three — integrative skills of listening and 
writing (SSQ 1.129). Thus, the shared variances of sound 
modality with total communicative proficiency were 11.2%, 9. OX 
and 8.5% totalling 28.7%. Factor Four was visual modality 
consisting of writing, reading, vocabulary, reading-writing, 
grammar and speaking (measured indirectly via reading). 

2. Vocabulary and grammar were grouped with the 
visual modality indicating that the test mainly measured 
this factor at this level of education. 

3. The visual modality factor shared common variance 
with the total variance of communicatively proficiency by 25%. 

4. The four factors could together explain domain- 
referenced communicative proficiency by 53.7% 

Factor analysis performed on test snore data of upper- 
secondary students( students aged 17-18) indicated that: 

** at this level of education, communicative 
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proficiency in English consisted of two factors. Factor One 
was sound modality consisting of listening, word-stress, and 
speaking assessed through a self-rating method. Factor Two 
was visual modality consisting of vocabulary and speaking 
measured indirectly via reading. 

16 

2. while vocabulary was grouped with visual 
variables, grammar was grouped with sound variables. Again the 
indirect test of speaking via reading was grouped under the 
visual modality factor. 

For university level (students aged 17/18-20/21) the 
findings indicated that; 

1. domain-referenced communicative proficiency was 
differentiated into three groups of factors with one sound 
modality factor and two visual modality factors. Factor One 
of the visual modality consisted of indirect speaking measured 
via reading, grammar, reading, and vocabulary, Factor Two of 
the visual modality consisted of integrative skills of reading- 
writing and unitary skill of reading. Factor Three was sound 
modality consisting of sound discrimination, word-stress, 
speaking measured via self-rating, integrative skills of 
listening and writing, and listening-reading-writing. 

2. visual modality factors combined could explain 
the variance of communicative proficiency by 20. 7% while sound 
modality could explain it by 31.5%. 

3. similar to the findings for the lower secondary 
level, grammar and vocabulary were grouped <ith visual modality. 
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4. simt lar to the f ind ings for both level 6 of 
secondary eduoation« the findings for this level confirmed the 
F/rouping of indirect speaking measured via reading with visual 
modality. (See Appendix V! for factor analysis results) 

Conclusions 

From the af orement i oned find i ngs for the three 

17 

levels of education the following conclusions about the nature 
and specific characteristics of communicative proficiency 
could be drawn! 

1» At the lower intermediate level of proficiency or 
the lower secondary level of education, sound modality factors 
were most finely differentiated. Conversely, at the upper 
intermediate level or tertiary level of education the visual 
modality factors k ere most finely differentiated* At the 
intermediate level of proficiency or upper secondary level of 
education each modality had equally only one factor. 

The findings might be the results of the nature or 
process of English acquisition of Thai students as well as the 
emphasis differently placed on each modality. At the lower 
intermediate level » oral-aural modes of communication are 
stressed most whereas at the upper- intermed i ate 1 eve 1 $ v i sua 1 
modality especially the reading skills receive most emphasis. 

2* Language component variables of sound* vocabulary 
and grammar wore never once grouped together into a 
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differentiated factor. They were always subsumed under the 
skill factors — sound discrimination and word-stress with aural 
/oral skills and grammar and vocabulary sometimes together 
under the visual nodal ity factor for lower secondary and 
tertiary levels , ar«i somet imes separately — vocabulary with 
visual modality and grammar with sound modality for upper 
secondary 1 eve 1 * 

3 . The sound factors across the three 1 eve Is of 
education could explain the variance of domain-referenced 
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communicative proficiency by 28. 7%, 36,9* and 31,5% respectively, 
while the visual modality factors could explain it by 25.0** 
9. IX and 20.7* respectively, or together at the ratio of 1,8)1, 
This illustrated the significant role of sound variables in 
student acquisition of communicative proficiency as they 
appeared to have more common variance with that of total 
commun i cat i ve prof i c i ency . 

LAM Gil AGE TEST I BO RESEARCH it THE Y%RD SEMSE 

This research also aimed at discovering the levels of 
stuient Bnlish communicative proficiency in the specified 
domains in the three levels of education. Parametric 
stat ist ical analyses were performed to find the mean, 
standard dev i at i on, kurtos i s , skewness and coef f i cient of 
variation (CV> of tho tests . The 5 criterion levels were used 
to identify of student lev? 1 s of communicative proficiency — 

1 ~ very poor* need max i ma 1 remed i a 1 work 
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2 - rather poor, need remedial work 

3 - average, still need remedial work 

4 = rather good, need some remedial work 

5 = very good, do not need any remedial work 

The findings indicated that for the lower secondary 
students their unitary skills of reading, listening, speaking 
were in general average while their integrated skills were 
quite weak. The CV indicated a high degree of score variation 
( 2-7). 

For the upper secondary students their speaking 

19 

skills were average while their other unitary and Integrated 
skills ranged from rather poor to average. The CV indicated 
an averagw degree of score variation (2-5). 

For university students, they were average to good in 
speaking, average in reading, rather poor to average in the 
integrated skills. The CV indicated an average degree of score 

variation ( 2-5) . 

Across the levels, Thai students ranged from levels 2- 
4 in their unitary skills while their integrated skills ranged 
from levels 2-3. The CV indicated that lower secondary 
students had the greatest degree of score variation (2-7) while 
the upper secondary and university groups' had average degree 
2-5 and 1-4 respectively. (See Appendix VII for students' levels 
of communicative proficiency in English) 
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RE CD ft MEM DAT IOIS 

The research on domain-referenced testing in both 
phases led to th* fol lowing recommendations! 

For Further Research 
1« Research into the degrees of contribution that 
language components have on acquisition of the unitary and 
integrated language skills should be conducted as well as 
research to find the most effective remedial and instructional 
processes for different levels of proficiency. 

2. Amplified objectivesi domain-referenced test 
tables of specification and domain-referenced test items should 
be developed, standardized and validated for every sub-level of 
primary, secondary, and tertiary education in Thailand. 

20 

For Practical Appl ications in English Teaching and Learning 

1. Language components — phonology, grammar, 
vocabulary — should not be presented or practised without the 
context of communication as they were never once grouped into 
a separate factor. 

2. At the beginning to lower intermediate levels or 
with students aged 7 to 14 the sound modality should get more 
emphasis than the visual one if language acquisition is to 
occur naturally. 

3. At the intermediate levels or with students aged 
15 t0 17 » ooth modalities may receive equal amount of 
presentation and practice. 

4. At the upper intermediate to advanced levels or 
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with students aged!7/18 to20/21 the unitary skill or integrated 
skills of reading and writing should be emphasized most. 

5. As the sound too da 1 i ty factors shared greater 
variance with communicative proficiency (at the ratio of UBl l) f 
media and materials for 1 istening, speaking, 1 istening- 
speak ing, 1 i stening-wr it ing f and 1 i steni ng-read i ng-wr i t i ng 
skills should be provided for both classrooom instruction and 
self-access learning for individual students in all levels of 
educat ion. 
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APPEMDIX I 

Amp (triad Object iVos for English Core Cours&i E 615$ 6 It 

(Upper Seconder j Level) 

1. After listening to the economic* political, and sports news, 
the student is able to answer four-choice multiple choice 

question about the main ideas and details of the news* 

2. After listening to the news, the student is able to make an 
oral summary of the news. 

3* While listening to descriptions of people* animals, objects 
and places, the student is able to ask for more information 
about the objects of the description. 

4. While listening to narrations about incidents such as an 
accident, the student is able to ta* e notes of what happened 
and able to sequence the incidents. 

5. After listening to songs and poems, the student is able to 
express his/her evaluative comments on what he/she has heard. 

6. When someone makes a wrong statement about something, e.g., 
a personal relationship, the student is able to correct it. 

7. The student is able to ask for details about people, 
objects, places, time and activities, and is able to 
answer Wh quest i ons accord ingly. 

8. The student is able to make an appropriate oral invitation 
to various groups of people on vart ous occas i ons . 
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9t The student is able to speak about, his determination to do 
something in an appropriate context or situation* etg., when 
asked about his/her future rjlan for study. 

10* When a clatfsmr*^e speaks about something in which the 

student is not interested* he/she is able to express his disinterest. 
11. After reading a letter or a note giving directions to a 
particular placet the student is able to draw a map of the place. 
12t After reading about a person's daily routine, the student 
is able to complete a time-table about that person's activities. 

13. After reading captions* the student is able to match the 
captions with the appropriate pictures. 

14. After reading an assigned short story or novel, the 
student is able to relate it to others. 

15. After reading a congratulatory letter or card, the student 
is able to write a thank-you reply. 

16. After listening to a telephone message, the student is 
able to take down the message. 

17. After reading a short story, an article or a novel, the 
student is able to summarize it. 

18. After reading a short story or a novel, the student is 
able to extend it from his imagination. 

19. After reading a "Want 11 ad, the student is able to write a 
1 ett er of app 1 i cat i on • 

20. After receiving good or bad news, the student is able to 

wr i te a letter of congratul at i on or sympathy to the person concerned . 
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APPEMDIX II 

Specimen Domain-Referenced Test Table of Sped float ion 



Ob J. 


Language 

Rtxncfc. i on 


Node 


Genre 


Channel 


Style 


Role 


Status 


Topic 


1 


Gett ing 
inf orna- 
t ion 


List- 
Speak 


News 


Radio/ 
TV 


Forma 1 


Stu* to 
Peers & 
Teachers 


Equa 1 


Bcont 

Pol it a 

Sports 


5 


Express 
Opinion 


Speak/ 
Writ, 


Liter 
ary 


Face to 

Face/ 

Radio 


Forma 1 


Student 
to Peers 
&Teacher 


Equa 1 


Point 
of view 


17 


Suminar i ze 


Read/ 
Writ. 


Short 
story 
&No- 
ve 1 

i 


Book/ 
Magaz ine 


Forma 1 


Student 
to Peers 
&Teacher 


Equa 1 


Adven- 
ture 



Test J ten Type and Test Technique 
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Item 1 Selection type — Four-choice Multiple Choice 

Length of News — 50-100 words 
Item 5 Supply type — Integrative skills — Listening/Speaking 

and Listening/Writing 

Length of Input (song and poem) 150-200 words 
Length of Output ( Student expression) 100-150 words 
Item 17 Supply type — Integrative skills — Reading/Writing 

Length of Input (Short story and article ) 1000-2000 words 



(Novel) 10000-15000 words 
Length of Output (Student summaries) Fo* short story 
nove 1 --500-600 words 

For novel — 1000-1500 words 



Marking Scheme 

Item 1 1 for correct 

o for incorrect 

Item 5 On a Rating Scale of 7 from l=*no competence» 
2=minimal competence, 3=thresho Id competence 
4= some communicative competence 
S^quite competent 

6=good-very good communicative competence 
7= ! near educated native communicative competence 
Item 17 Same as Item 5 
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APPSBDIX III 
Formulae for I torn Analyses 

1. Whitney & Sabers Multiple-point Item Analysis 
Facility Value (FV> - S + S - < N ) X 4 

%x % T win 

N (X 

T mm h - X» i n ) 

S = total scores of the upper group 

S * total scores of the lower group 
i 

N = total number of testees used in the analysis 

T 

X ^highest score obtained 

tm u m 

X = 1 owest score obtained 

« t n 

D i scr i m i nat i on I ndex ( D ) 
D = S - S 
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N (X - X ) 



S fS t X ,X symbolize the same values as in the 

formula for FV 

- number of te&Lees in the upper group 
Sources Whitney ?< Sabers, 1970, 1971 In Scannell & Tracy» 
1975: 221-224. 

2, Domain-Ref ©renced Test item Analysis 

Fac i lity Val ue ( FV) = Proportion of testees 
passing the each item after instruction 



Sensitivity IndexCS f )= R post % ~ R pre i 



R post ^ - Number of testees passing item^ after 
i nst ruct i on 

R pre ^ = Number of testees passing the item ( i tem ) 

fe m r q r m » nitruct ion 

T * Total number of testees 
The values of ran?e between -1 and +1. Good 

items should have the values close to +1. 
Sources Pitiyanuwat, 1986i 79-80. 

3. Index of Item-Objective Congruence eContent Validity) 
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1 = ( k - 1 )s - S 

* © o o 1 



2N (k - 1) 

I = Index of item objective congruence for item and 

to * 

Object i ve 

K " Number of objectives to be measured by the test 
N = Number of spec i a 1 i st raters 
S * Total scores of all raters for objective 
S tt Total scores of all raters for all 

object i ves except ob ject \ ve 

o 

Sources Wibulsri, 1983. 



4. Swaminathan, Hambleton & Algina Coefficient of 
Rel iabi 1 !t,y 

K = P ~ P 



1 - P 

m 

K * Coefficient of Reliability 

P = Observed Proportion (Proportion of 
testees passing both pre and post tests + proportion of 
testees failing both pre and post tests) 

P » Expected Proportion E (Proportion of 
testees passing both tests + Proportion of testees passing 
the pre- test but f a i 1 ing the post test > x ( Pro port i on of 
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testees passing both tests + Proportion of testees failing 1 
the pre test but passing the post test ) 
(Proportion of iestees failing the pre test but passing the post test) 



(Proportion of testees failing both tests) x (Proportion of 
testees passing the pre test but failing the post test + 
Proportion of testees failing both tests)] 

Examples Criterion of Passing = 80% 



Pre 
Post 



Pass 



Fai 1 



Total Pass 



Pass 
Fai 1 
Total 



.66 
.04 
.70 



. 14 
. 16 
. 30 



.80 
.20 
1.00 



P = , 66 + . 16 = . 82 

P = (.70) (.80) + (.30) + (.20) = .62 
K = .82-. 62 



1-. 62 
- .53 

Domain-Referenced Validity Coefficient 
V = a + c 



3.9 
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a + fa + c + d 
V ~ Validity Coefficient 

a = Number of people passing the criterion after 
instruct ion 

b - Number of people passing the criterion before 
instruct i on 

c 3 Number of people not passing the criterion before 
instruct i on 

d = Number of people not passing the criterion after 
i nst ruct i on 

Val idity coefficient ranges from 0 1, A 

value close to 1 indicates test val idity. 
Sources Pitiyanuwat, 1986s 79-81 



APPEMDIX tV 
The Tosh Domain-Content Stsdartf Scores 

Symbol s used in the table 
S = Speaking 
L = Listening 
R = Reading 

ES^Essay or paragraph writing 
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*0r. 

Dl=Dictation ( L i stenin*~Wr i t ing > 
RW= Readintf-Writing 
LW=L i stenin^-Wr it in? 

LR = L i stening-Read intf-Wr it in? (Partial Dictation) 

L ■ Lower Level of Secondary Education 
U = Upper Level of Secondary Education 
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The Table reveals that for lower secondary students, the 
items grouped into high, middle and low percent i les measured 
the same skills by 40%, 65%, and 07 % respectively for Forms A, B 
and C averaging 57% of the items measuring the same skills for 
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the three forms. 

For the upper level of secondary education* the test 
items in the three forms measured the same skills by 49% on the 
average* 

Students in both levels of secondary education could 
do items measuring reading and rending best, second were items 
measur ing t he i r speak ing skills via a se] f -rat ing method . The 
items they could do least were items measuring their 
integrative skills of 1 i stening-wr it ing t listening-reading- 
wring, reading-writing and essay/paragraph writing* 



APPSMD/X V 

F-tests of the Domain-Referenced Tests' ICC (Item Characteristic 
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Curret with the Standard ICC 



Lower Secondary Level 



Form 


X 


SD 


N 


A 


0.02 


1 . 49 


42 


B 


0.04 


1 .24 


42 


C 


0.04 


1 .30 


39 




SS 


DF 


MS 


Bet. 








Groups 


. 10 


2 


.00 



Within 

Group 211.94 120 
Total 211.95 122 



1.81 



.00 0.99 



Upper Secondary Level 



Form 


X 


SD 




N 


A 


0.06 


0.91 




35 


B 


-0.02 


1.28 




36 


C 


0. 10 


1.05 




36 


SS 


BF 


MS 


F 


P 


.23 


2 


0. 1 1 


0.10 


0.99 



117. 44 
117. 67 



104 
106 



1.20 




44 



Abbrev i at i ons 



APPEMDIX VI 
factor Analysis Results 

FV = Factor for Visual Modalities — Reading, 
Writing, Reading-Writing, Writing, 

Indirect Speaking Measured via Reading 

FS = Factor for Sound Modalities — Listening, 

Listening-Reading-Writing, Li steninj-Wr it ing 

SD - Sound Discrimination, ST=Word Stress 
VOC = Knowledge of vocabulary, G = Use of Grammar 

1S(R) = Indirect Speaking Measured via Reading 



Lower Secondary 

( N s 679) 

FV I FS I FS I I FS III 

W L L-R-W L-W 

R 



ST 

VOC S 

R-W 

JSCR) 

G 
SSQ 

2.54 1.458 



; D 



1.193 1.128 



Upper Secondary 



( N 


525) 




(N*493> 




FS I 


FV I 


FSI 


FVI 


FVI! 


L 


Voc 


SD 


1SCR) 


R-W 


ST 


R 


ST 


G 


W 


G 


IS(R) 


S 


R 




S 


R-W 


L-W 


VOC 




L-W 


W 


L=R-W 






L-R-W 










SSQ 




SSQ 







3.256 



2.447 



Tert i ary 



2.339 1.974 1.426 



APPSMDfX V J I 

Students' Levets of English Proficiency 
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* w 



English Proficiency Lower Secondary Upper Secondary Tertiary 

Level CV Level CV Level CV 



Language Components 3 1 2-3 2 3 1 

Sound Modalities 2-3 13 2 3 2 

Visual Modalities 2-3 2 2-3 2 2-3 2 

Components&Modal i- 2-3 1-7 2-3 2-5 3 1-4 
ties Combined 
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